A SYSTEM AND METHOD FOR USING PRINTED DOCUMENTS 



1 BACKGROUND 

2 The design of displays for computing devices is always a compromise. If the 

3 display is large, then the device is too expensive, and too large, to be practical. If the 

4 display is small, then too little information can be displayed at once. Thus, when viewing 

5 large documents, users can only view a small portion at any one time, and must often 

6 manipulate controls to adjust the visible portion - looking first at one part, then at 

7 another. This works, but is inefficient. 

8 Printed documents are easier to read than most computer displays. They can be 

9 much larger and have higher resolution than computer displays. They can also be 

10 navigated quickly - for example, a salesperson familiar with the company catalogue can 

1 1 very quickly locate the page where an item of interest is described. However, printed 

12 documents have the disadvantage that they are static, and often out-of-date. 

1 3 One way of interacting with a printed paper is described by Anoto AB, a 

14 subsidiary of C Technologies AB of Sweden. They print special paper forms, which have 

15 a specific address printed in each 2mm x 2mm square section. The Anoto™ system uses 

1 6 a pen with a built-in camera that recognizes the printed pattern. If a user writes on the 

17 paper with this pen, the pen reads the addressing information, and sends the addresses to a 

1 8 server. The server forwards the pen's recorded motion to the entity that has 

1 9 leased/purchased the identified portion of the fixed address space. For example, suppose 

20 Filofax Group Limited purchases a section of the address space for e-mail purposes. A 

21 user could then buy e-mail paper from Filofax®. When the user writes on the e-mail 

22 paper and checks the "send" box, the pen sends its recorded movements (a list of the 

23 addresses on the page over which it passed) to the Anoto™ server. The server looks up 

24 the addresses, sees that they were leased to Filofax, and forwards the recorded 

25 movements to that company's server. The Filofax server then translates the pen's 

26 motion into an electronic message, e.g., e-mail, and then e-mails it to the user. 

27 A disadvantage of the Anoto™ system is that the encoded address is of a fixed 

28 size. Each location on one of their printed pages corresponds to an absolute location in a 

29 60 million km 2 grid. This works well for printed forms (where a small section of the 

30 address can be repeatedly used on thousands of identically-printed documents), but it 

3 1 doesn't allow for user-printed documents. Further, the Anoto™ system patterns must be 

32 printed on a high resolution printer at approximately 1 000dpi to fit the whole address into 
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1 a 2mm x 2mm square. Also, the system uses the combination of pen/printed paper only 

2 as an input device, and the method they employ can only provide positional information 

3 while the pen can see the paper. It does not provide a means to display up-to-date 

4 information to the user, and if you slide off the edge of the page, it stops working. 

5 Several mechanisms are presently available for encoding an address on printed 

6 media that identifies the printed document or points to an address, /. e. , Uniform Resource 

7 Identifier (URI). For example, one could print a barcode, or other identifying mark, on 

8 the document. Then one could read the barcode, recall the document, and then explore it 

9 using a pointing device (such as a mouse). 

10 One can also use printed bar codes to serve as input devices. For example, in a 

1 1 store, the clerk can scan a printed bar code, rather than having to manually type an item 

12 number. It is even possible to encode large amounts of information in printed marks, and 

13 then extract it by scanning. Examples include the dataglyph developed by Xerox 

14 Corporation, and the digital watermark system developed by Digimark. 

1 5 While there are numerous devices and methods for interacting with printed 

16 documents, there is a need for devices and mechanisms which allow the printed document 

17 to replace both large keyboards, and large display devices, in a very natural way. 

18 SUMMARY 

19 It is desired to combine the benefits of a computer display (providing immediate 

20 access to up-to-date information) with the benefits of a paper document (providing a 

21 large, high-resolution display area). It is also desirable to use printed documents to 

22 provide large amounts of relatively static information, while using computing devices to 

23 provide relatively small amounts of up-to-date information. 

24 Furthermore, is would be desirable to use the same printed documents to 

25 supplement the input mechanisms on the computing device, so that the combination of the 

26 printed document and the small device serves as a replacement both for a large display, 

27 and a large keyboard. 

28 An embodiment of the present invention uses specially printed paper to 

29 supplement, or replace, the large displays and user interfaces commonly required for 

30 efficient use of computing devices. An embodiment provides a method which requires 

3 1 only a single sensor and which supports a wide range of operations. It is shown that a 

32 combination of a small computing device, and a printed page, can be as efficient as a full 

33 sized computer with a large display and keyboard. 
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An embodiment of the present invention overcomes the disadvantages of the prior 
art by providing, on a printed page, an encoded message in a way so that multiple copies 
are distributed across the whole page and so that the message has an arbitrarily large size. 
In that way, the message may be unique to every new document that is printed and may 
include a full web address and/or security features. Since the message appears in multiple 
locations, the whole message may be obtained, even if parts of the document are missing. 
Both the message and the absolute location are encoded in the same way, so they can be 
read with a single image sensor, and with a single motion of the sensor over the paper. 

Furthermore, the same sensor may be used to detect the relative motion of the 
device. As the sensor moves, it may use images of the surface below to detect relative 
motion. By comparing its measured relative motion, with the absolute position 
information encoded on the page, it is able to calibrate its motion sensor to the page, and 
may then continue to provide relative location information even when the sensor is slid 
past the edge of the paper. In this way, the user may navigate beyond the bounds of the 
real paper document. 

In an exemplary embodiment, the combination of a handheld device and a 
specially printed page are used to plan a road trip. In this case, the printed page appears 
as a map to the human user, but contains both encoded location information, and a 
message instructing a handheld scanning device that it is a map, and how to interpret the 
location data. The handheld device includes a graphical display which may be transparent 
or opaque, and an image sensor on the underside. The image sensor is used both to read 
the encoded information and to detect relative motion of the device. As the user slides a 
PDA (personal digital assistant, or other portable device) across the map, it extracts the 
message, and begins to show details of the cities/sights related to the region of the printed 
page beneath it. To plan the trip, they move over sights of interest and press a "select" 
button when each is in the display. In this way, the combination of the printed page and 
the handheld device is used to quickly enter large amounts of information in a very 
natural way. When done, they slide to a region of the map on which is printed "plan my 
route". The PDA then plans the route, and displays it. If the user wishes, they can now 
view the route by sliding back over the map - on the screen they see the map below, with 
their route overlaid on top. In this way, the combination of the printed page and the 
handheld device is also serving as a replacement for a huge display. The user may then 
leave the printed map behind, and still manipulate the on-screen map, merely by sliding 
the hand-held device on any convenient flat surface. 
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DESCRIPTION OF DRAWINGS 

An exemplary embodiment of the invention is illustrated in the drawings in which 
like numeral references refer to like elements, and wherein: 

FIGURE 1 is a representation of a printed page encoded with an address 
according to an embodiment of the present invention; 

FIGURE 2 is an illustration of a printed sales by region map as might be used in 
an application of an embodiment of the present invention; 

FIGURE 3 is an illustration of an application of an embodiment of the present 
invention where a personal digital assistant acts as a magnifying glass for a printed sales 
region map document; 

FIGURE 4 is an illustration of a calendar page as might be used in an application 
of an embodiment of the present invention; 

FIGURE 5 is an illustration of an application of an embodiment of the present 
invention where a personal digital assistant acts as a magnifying glass for a printed 
calendar document; 

FIGURE 6 is a block diagram showing a system according to an embodiment of 
the invention; and 

FIGURE 7 is a flow diagram showing the method of an embodiment of the 
present invention. 
DETAILED DESCRIPTION 

The numerous innovative teachings of the present application will be described 
with particular reference to the presently preferred exemplary embodiments. However, it 
should be understood that this class of embodiments provides only a few examples of the 
many advantageous uses of the innovative teachings herein. In general, statements made 
in the specification of the present application do not necessarily delimit any of the various 
claimed inventions. Moreover, some statements may apply to some inventive features but 
not to others. 

In an exemplary embodiment of the present invention, the system includes a 
printed page containing encoded information, a hand-held scanning device which can 
both read the encoded information and detect relative motion, a computing device (with 
access to a storage device) which can perform operations in response to the encoded 
information and measured motion, and display and user interface components for 
additional interaction with the user. 
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1 The information encoded on the printed page includes both a digital message, and 

2 absolute location information. These may be encoded by printing a set of dots on the 

3 page. In some aspects of the invention, the dots are invisible to the naked eye. One 

4 method for encoding the message and absolute location is to print the dots by moving 

5 them very slightly away from a perfect position. The amount and direction that the dot is 

6 moved away from that perfect position is the way information is encoded. An alternative 

7 way to print the dots is by printing dots and known spaces, such that a blank spot means 

8 there isn't information and a spot is put where there is information, as in a binary system. 

9 Another method for printing dots on a page is to merely print the actual characters 

1 0 of the message and location in an ink which is not visible to the human eye but which is 

1 1 visible to the camera. Then the camera reads the characters using standard optical 

12 character recognition techniques. An ink which is visible in the near infrared might be 

1 3 used for this method. It will be apparent to one skilled in the art how to encode the 

1 4 message and location on the page using a variety of methods so that it can be made 

1 5 machine readable. 

1 6 Referring now to the drawings, and in particular to FIGURE 1 , there is shown a 

1 7 page with a representation of an encoded message. In this exemplary embodiment, the 

1 8 message corresponding to the printed document is "ABC." Fragments of the message, 

19 e.g. , "A", "B" and "C" are encoded at different pseudo-random locations on the page. 

20 For instance, the "A" portion of the message is encoded at points 101 and 103. The "B" 

21 portion of the message is located at points 105, 107 and 109. The "C" portion of the 

22 message is located at points 1 1 0, 1 1 3, 1 1 5 and 1 1 7. Absolute location information is 

23 also encoded on the printed page. Referring again to FIGURE 1 , the upper left corner 

24 region is encoded with a "0,0" at 1 1 9; a center region is encoded with a " 1 , 1 " at 1 2 1 ; etc. 

25 In the exemplary embodiment, the fixed mapping between absolute locations and 

26 message fragments appears in Table 1 . 
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Absolute Location 


Fragment Position 


0,0 


3 


1,0 


1 


2,0 


2 


0,1 


2 


1,1 


2 


2,1 


3 


0,2 


1 


1,2 


3 


2,2 


3 



1 TABLE 1. Mapping between Absolute Locations and Message Fragments 

2 In one embodiment, the printed page is scanned by a mobile, possibly handheld, 

3 device which reads a swath of the printed page. Referring again to FIGURE 1 , a swath is 

4 shown 1 50 by a dashed line. In this exemplary embodiment, a camera or scanner is 

5 moved across the page in the path indicated by the dashed lines 150. For this movement, 

6 the message fragments and absolute locations read would be (0,0), (1,1) and then (1,0). 

7 While the fragments are read in the order of CBA, they are decoded as ABC because each 

8 fragment is mapped to an absolute location and the device reads the absolute location and 

9 decodes it using a table such as TABLE 1 , above. 

10 When the page is scanned in its entirety, the scanning device will have all of the 

1 1 message information at its disposal. If a portion of the page is missing or damaged, the 

12 message is still often retrievable due to the repetition of message fragments A, B and C 

13 101, 103, 105, 107, 109, 1 10, 1 13, 1 15 and 1 17 across the page. It will be apparent to 

14 one skilled in the art that various schemes can be used to determine how frequently and at 

1 5 what resolution the message fragments appear on the printed page. There are obvious 

1 6 trade-offs regarding printer and scanner resolution, visibility to a human eye, range of 

1 7 motion for a handheld device to retrieve an entire message, etc. One skilled in the art will 

1 8 determine the requirements necessary for the desired application and adjust the trade-offs 

1 9 accordingly. 

20 In the exemplary embodiment, the handheld device contains an imaging device 

2 1 capable of both reading the encoded information and of determining the relative motion 

22 of the handheld device. One way to implement this is by using a small digital camera and 

23 lens such that the camera is focused on a small spot on the surface on which the device 
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1 rests. The camera may detect the encoded information by using standard computer vision 

2 techniques to recognize the printed dots in the image, and extract the information from 

3 their measured locations. The camera may also detect the relative motion of the device 

4 by taking images of whatever surface the scanner happens to be on, and then using 

5 standard computer vision techniques, analyzing the changes in the images over time to 

6 determine relative translation and rotation in the plane of the surface (this is the similar to 

7 the way current optical mice operate). 

8 As the device slides across a printed page, it can calibrate its measured motion by 

9 comparing the measured relative motion to the observed printed absolute locations. Then 

1 0 it can use those calibration parameters, along with its measured motion, to compute a 

1 1 relative location. For example, say the device notices that every 10 cm of horizontal 

12 motion corresponds to 5 units of printed absolute location, then if you moved 30 cm to 

1 3 the right, it would compute a relative motion of 1 5 units to the right of where it started. 

14 Notice that so long as the device starts on the paper (to get an initial absolute location, 

1 5 and calibrate itself), it may continue to provide relevant information well beyond the 

16 limits of the printed page. The imaging device, and the computation to analyze the 

1 7 images, may reside in separate modules, or they may be implemented in a single self- 

1 8 contained module. The implementation of such systems will be apparent to one skilled in 

19 the state of the art. 

20 In the exemplary embodiment, the handheld device contains a display. That 

21 display may be implemented with existing Liquid Crystal Display (LCD) technology. 

22 This technology permits the display to be either transparent (in which case the displayed 

23 information may appear to overlay whatever surface is beneath the display) or opaque (in 

24 which case it looks like a standard computer notebook display). Those familiar with the 

25 state of the art should recognize that there are a number of alternative display 

26 technologies suitable for such use, and the choice will depend on a number of design 

27 trade-offs. It should also be apparent that there are alternative methods of providing 

28 information for the user. For example, the device may contain a printer. In one 

29 embodiment the printer is a standard office printer, physically separate from the handheld 

30 device. In another embodiment, the printer is a separate module located within the 

3 1 handheld device. In an alternative embodiment, the printer is capable of printing directly 

32 on the surface beneath the handheld device, so that it prints information onto the same 

33 page from which it is obtaining encoded positional information. In that case the device 
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1 may rely entirely on motions of the user's hand, and need have no moving parts (aside 

2 from the ink droplets). The device could also contain an audio or video playback module. 

3 In the exemplary embodiment, the handheld device contains a user interface. This 

4 may be implemented with push-buttons, dials or sliders. There are trade-offs in deciding 

5 how many controls, and of which types, should be included. Those familiar with the state 

6 of the art will be able to make such choices based on the particular applications intended 

7 for the device. 

8 In the exemplary embodiment, the handheld device also includes a computer. 

9 This computer recognizes messages detected by the scanner and performs pre-coded 

1 0 actions which depend on the encoded message, the sensed location information (both 

1 1 absolute and relative), the state of the user interface (is the user pressing any button) and 

12 its previously-stored state. For example, the combination of this handheld device and a 

1 3 specially printed page may be used to plan a road trip. In this case, the printed page 

14 appears as a map to the human user, but contains both a message and encoded location 

1 5 information. As the user slides the PDA across the map, the encoded message is 

16 extracted, and the computer recognizes that it refers to a particular map. Then it may 

17 combine the absolute and relative location information to extract from an internal 

1 8 database a list of cities/sights related to the region of the printed page beneath the device. 

1 9 These may then be displayed for the user. To plan the trip, the user moves the device 

20 over sights of interest and presses a "select" button when each is in the display. Each 

21 time this is done, the computer remembers the currently-displayed city/sight information 

22 and its location. In this way, the combination of the printed page and the handheld 

23 device is used to quickly enter large amounts of information in a very natural way. When 

24 done, the user slides to a region of the map on which is printed "plan my route." The 

25 computer recognizes the combination of the message and this absolute location, so it 

26 plans a route through all the remembered locations. Now, it may display the route for the 

27 user. Each time the device is moved, it determines the new location, and draws the 

28 portion of the route corresponding to the portion of the map under the device. If the user 

29 wishes, they can now view the route by sliding back over the map - on the screen they 

30 see the map below, with their route overlaid on top. The user may then continue to 

3 1 manipulate the on-screen map, merely by sliding the hand-held device on any convenient 

32 flat surface. In that case the computer notices that no absolute position is available, and 

33 uses the relative location information and the last known absolute position. The user also 

34 has the option to zoom in or out, changing the scale of the computer-generated display 
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and also the relation between motion of the device and motion of items in the display. It 
should be apparent to one skilled in the art that this relation may be computed such that it 
appears to the user as though the display were a window over a virtual document, and 
where the computer zoom control has the effect of enlarging or shrinking that virtual 
document. 

In one embodiment, security or authorization information is contained within the 
message. When a document with security information encoded within the message is 
scanned, a user may be required to enter a password, or the identity of the user may be 
assumed depending on the access device being used. For instance, the user may have 
already logged on and identified oneself. 

In an alternative embodiment, additional information may be encoded in the 
message for marketing purposes. For instance, when sending each customer a document, 
a company embeds the customer's ID number within each printed copy. Then, when a 
device with access to the Internet is used on one of those documents, it (with the user's 
permission) notifies the company that a particular customer's printed copy is being 
accessed by a particular device. This, for example, enables a company to track how many 
different people use each particular printed copy. 

In another embodiment, the message comprises an embedded sequence of 
processing instructions, or macro. The macro uses one or both of the absolute and 
relative location information as arguments. Once the macro is scanned and extracted, the 
processing instructions are interpreted, or executed, by the computing device. In an 
alternative embodiment using macros, the macro holds instructions to retrieve or modify 
data in at least one storage device and uses that obtained information in its execution. For 
example, the system may store the state of the user interface, or it may retrieve 
information for display from a database on a storage device. 

In another embodiment, a device is used to retrieve or display more detailed 
information related to an absolute location on the printed page. Referring now to 
FIGURE 2, a sales region 200 for a particular market is represented. Four distribution 
centers, or sales offices are shown 201, 203, 205 And 207. Suppose a salesperson has this 
printed page, but not a printed version of the entire inventory list of each center while on 
a sales call. If additional information is needed, it can be produced in the field using an 
embodiment of the present invention. The salesperson has a specialized PDA with a 
camera or scanning device located typically on the underside. It will be apparent to one 
skilled in the art that the location of the camera or scanning device will depend on the 
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1 ergo-dynamics of the handheld device. FIGURE 3 shows a specialized PDA 301 on top 

2 of the printed page with the sales region 200. 

3 Referring now to FIGURE 3, the PDA 301 is placed over sales office 203, which 

4 happens to represent New York City, NY. The PDA will then display information 

5 related to the region, such as the name of the sales office 303, a volume of business 305, 

6 an average price, and a quantity of customers 307. It will be apparent to one skilled in the 

7 art that the displayed information is dependent on the desired application and is defined 

8 by an application programmer. In some embodiments, an action by the user may be 

9 required, such as a mouse or button click/press, before displaying the related information. 

10 This action will avoid visual confusion as the display updates with varying related data 

1 1 continuously as the PDA or other device is moved across the printed page. 

12 In another embodiment, the system is used to access updated calendar 

13 information, rather than related information. Referring now to FIGURE 4, there is shown 

14 a representation of a calendar 400 for a selected week. The calendar has items stored for 

1 5 each day of the week. The calendar is accurate at the time of printing, but may be 

16 updated a number of times without reprinting. In some cases, the owner of the calendar 

17 will update the items, but have no access to a printer, for instance while on business 

1 8 travel. Referring now to FIGURE 5, the PDA 301 is placed over a day of the week, e.g., 

19 Thursday 501. While the printed version of Thursday's events 400 shows only one item 

20 401, the version displayed on the PDA 301 is updated to show four (4) items 503. 

2 1 Furthermore, if the device is now slid down off the printed calendar it will display the 

22 following week, if slid further, it will display the week after that, and so on. The user 

23 may now dispense with the printed page entirely, zooming in and out using controls, and 

24 sliding up/down left/right to move through different calendar entries, days, weeks, 

25 months, and years. It will be apparent to one skilled in the art that the entire user's 

26 calendar may be accessed in this way and that the same techniques are applicable to a 

27 range of other domains. 

28 In another embodiment, there is a system where every salesperson has a 

29 specialized PDA with a memory card in it. The memory card holds the company 

30 confidential information about the cost of each product and current availability and other 

3 1 relevant data. The device has all of the information built into it and the salesperson just 

32 carries it around with them. An advantage of this embodiment of the present invention is 

33 that the salesperson gains access to information inside the device in a way that's very 

34 natural. For example, the salesperson has a printed catalog with a picture of every 
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1 different product that they sell and they flip open to the page where the particular product 

2 is that their customer is interested. Then the salesperson just runs across that page with 

3 their PDA and up on the screen of the PDA pops the most up-to-date information, how or 

4 if any of those products are available and what the current price is and what the backlog 

5 is, etc. The printed page serves as an interface for relatively static information and the 

6 PDA serves as your interface for the more dynamic portion of the information. 

7 In this scenario, a printed page might have one corner in which is printed a blank 

8 box that said "availability" but doesn't have any number in it. In the other corner a blank 

9 box is printed that says "costs." When a user slides over the cost box with the PDA it 

1 0 shows the cost. When a user slides over the availability box, the PDA display shows the 

1 1 current availability. In one embodiment, this information is maintained locally, within 

12 the PDA. An advantage of this embodiment is that no communication is necessary to 

13 other servers or systems in a network. The salesperson has another printed page, 

14 containing a grid. From left to right are the different industry sectors. Under each sector 

15 heading is a copy of the alphabet, from A at the top, to Z at the bottom. To access 

1 6 information on a particular customer, the salesperson simply slides their PDA across the 

17 page to the sector they're interested in, and then slides up and down to scroll through all 

18 their customers. They can get to any particular customer quickly by sliding straight to the 

1 9 appropriate letter of the alphabet. In this way, the salesperson is able to navigate both the 

20 product database, and the customer database, using a single device, and without any user 

21 action other than sliding the device on a printed page. Furthermore, since the message 

22 encoded on the page is encoded redundantly, the salesperson need not carry the whole of 

23 each printed page with them. Once familiar with the customer lists and the product 

24 catalogue, they could print a single page, with a small portion of the customer list, and a 

25 small portion of the catalogue. Now, to access one or the other, they slide over that small 

26 portion, then raise the device, place it on a table, and then continue to slide. On the 

27 screen, the result is exactly the same as when they had the whole printed page beneath the 

28 device. 

29 The user (salesperson) has all the relevant data local to the printed page. In other 

30 embodiments, the data is retrievable over a network, by wired or wireless methods. Other 

3 1 embodiments may store some data locally, and some remotely. 

32 A system 600 utilizing an embodiment of the present invention is shown in 

33 FIGURE 6. A storage or memory 601 holds a plurality of documents and/or related data. 

34 The storage is connected to a printing device 605, typically through a CPU or other 
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1 computing device 603. In some embodiments, the printing device has an imbedded 

2 processor and accesses the storage 601 directly. The printing device is capable of 

3 printing documents from the storage encoded with message and absolute location 

4 information 65 1 on a page 650. It will be apparent to one skilled in the art that the 

5 specifications of this printing device are dependent on the method selected for encoding 

6 the information. For instance, if it is desired that the address be invisible to the naked 

7 eye, the printing device 605 might use a special ink. Various resolutions may be 

8 implemented based on trade-offs, as discussed above. 

9 In one embodiment, a handheld device comprises a CPU 6 1 0, storage 611, user 

10 interface 612, display 613, an image sensor 613, and an image analyzer 614. The image 

1 1 sensor takes images of the surface on which the device is located. The image analyzer, 

12 analyzes those images to extract a message and absolute location (when the handheld 

13 device is over a printed document 650) and relative motion (regardless of the surface on 

14 which the device is located). When a message is recognized by the image analyzer, it 

1 5 alerts the CPU 6 1 0, which looks up the message in its storage 6 1 1 , and performs the 

1 6 appropriate pre-programmed action(s). The user may alter those actions using the user 

17 interface 612, and may observe the results or side effects by viewing the display 613. 

18 In some embodiments, the storage 611 may be supplemented or replaced by 

19 remote storage 601 via a remote connection 607. The connection to remote storage 601 

20 can be via a local area network (LAN), wide area network (WAN), global distributed 

21 network, intranet, extranet or wireless network. 

22 In some embodiments the various components may not all be physically 

23 connected, but may reside in separate containers connected via wired or wireless means 

24 using dedicated or shared connections. For example the image sensor could be located in 

25 a pen, while the image analyzer, CPU and display were located in a PDA. 

26 FIGURE 7 shows a flow diagram which describes a method for using an 

27 embodiment of the present invention. A document is printed in block 70 1 using a special 

28 printer, where the document is encoded with message and location information, as 

29 discussed above. This document will typically have useful data imprinted on it in 

30 addition to the message and location information. If a user desires to interact with the 

3 1 printed document, then they slide the imaging device across it, in block 702. As the 

32 device moves, it continuously computes relative motion in block 703. 
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1 While the device is over the printed page, the image analysis system detects the 

2 presence of the encoding and extracts a fragment of the message, and an absolute location 

3 from the image in block 704. 

4 As the device moves across the page, more and more of the message is collected, 

5 until the entire message has been accumulated. A determination is made as to whether 

6 the entire message has been collected in block 705. If the entire message has been 

7 collected, the system retrieves the pre-coded action corresponding to that particular 

8 message in block 706. 

9 In cases where the device is able to extract the absolute location information in 

1 0 bl ock 704, it uses that knowledge, in block 707 to calibrate the measured relative motion. 

1 1 The last-known absolute location is used with the measured relative motion to compute a 

12 new location. The pre-coded action is then executed based on the location computed, in 

13 block 708. 

1 4 Finally, the results of the pre-coded action are displayed for the user in block 709. 

1 5 The user may then continue to move the device, and the display will continuously update. 
!6 Executing the pre-coded action may require the device to retrieve related data 

1 7 from a local or remote location based on the message and application. If the message is 

1 8 defined with security or authorization levels, the user is authenticated prior to the display 

1 9 or output of any data. It will be apparent to one skilled in the art that any of a variety of 

20 authentication methods, now known or to be invented, may be used to enable this 

21 functionality. 

22 The terms and descriptions used herein are set forth by way of illustration only 

23 and are not meant as limitations. Those skilled in the art will recognize that many 

24 variations are possible within the spirit and scope of the invention as defined in the 

25 following claims, and their equivalents, in which all terms are to be understood in their 

26 broadest possible sense unless otherwise indicated. 
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